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^ ■ The measurements of rare decays are highly sensitive to physics beyond the standard model. 

In this article limits on the branching ratios of the decays B^^-. -^ fi'^ fj,~ , D" — > fj-'^ ij.~ and 
D^ — > TY^ fi'^ fj,~ are presented. Furthermore the first measurement of the branching fraction 
r — ^ ' and CP asymmetry of A° — > pii^ and A'^ — » pK" decays is described. Data samples with an 

integrated luminosity of up to 2 fb~^ collected at the Tevatron pp-coUider at y^ = 1.96 TeV 
were used in these analyses. The results are consistent with the standard model predictions 

i. and tighten the constraints on new physics models. 

D 

Pi, 1 Introduction 

The branching ratio of a rare decay mode is an interesting quantity to measure because the 
contribution from physics beyond the standard model, which can be neghgible compared to the 
^ ' standard model (SM) contribution in the dominant decay modes, may be sizable in the rare 

£J2 ■ decay mode. Decays are suppressed in the SM for different reasons. One of them is that flavor- 

r^^ I changing neutral current (FCNC) processes are forbidden at tree level. The FCNC decays 

CN ■ discussed here are iJP,-, — > /i'^/x~, D^ -^ /^^A*" and Df. -^ 7r+/i+/i~. Although the decays 

^' ' A^ -^ pTT~ and A^ — > pK~ are allowed at tree level in the SM, they are suppressed because the 

6 to u quark transition involves the small CKM matrix element Vub- 
(^ [ In order to be able to observe rare heavy flavor decays it is essential to produce a sufficient 

amount of bottom and charm hadrons. The large bb and cc cross section at the Tevatron, 
more than four orders of magnitude higher than at the B-factories, allows to probe very small 
branching ratios. Another advantage of the Tevatron is the production of all species of b hadrons 
C^ I so that rare decays of B^ mesons and b baryons can be studied. 

On the other hand the inelastic cross section is 10^ times higher than a{bb) requiring very 
selective and efficient triggers. For rare decays in particular triggers on pairs of muons or 
pairs of displaced tracks are used. Another challenge at the Tevatron is the high combinatorial 
background from fragmentation tracks. Sophisticated selection procedures based on kinematic, 
topologic and particle identification quantities are employed to extract the signal. 

2 Rare decay measurements 

2.1 Bf^^^fi+fi- 

The FCNC process B^ -^ /^^M^ is predicted to have a branching ratio of 13{B'^ — > ^+/i~) = 
(3.42 lb 0.54) X 10~^ in the SM^. The B^ — > jjl'^ ii^ decay is further suppressed compared to 
the B'^ decay by |14d/^sP- The SM prediction for the branching ratio is B{B^ -^ fJ-^l^^) = 
(1.00 lb 0.14) X 10"^''. A significant enhancement of the B'^ and B^ branching ratios is predicted 
by several new physics models. For example in the minimal super-symmetric standard model 
(MSSM) the B'^ branching ratio is proportional to tan^/3 where tan/3 is the ratio between the 
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vacuum expectation values of the two neutral Higgs fields. In i?-parity violating super-symmetric 
(SUSY) models an enhancement is possible even at low values of tan /3. 

Both Tevatron experiments optimize the selection of B'^ — > ^'^ n~ candidates using simulated 
signal events and background events from mass sidebands. While DO combines the discriminant 
variables in a likelihood ratio, CDF uses a neural network (NN). It was checked on background 
samples that the NN does not introduce a selection bias. Both experiments estimate the combi- 
natorial background by a fit to the mass sidebands. The contribution from decays of B mesons 
to two light hadrons, which could peak in the signal mass region, was estimated to be an order 
of magnitude lower than the combinatorial background. To obtain an absolute branching ratio 
the number of signal events is normalized to the high-statistics B^ -^ J/ipK~^ mode. For the 
limit calculation CDF splits the data sample in three bins in NN output and five bins in mass 
which improves the sensitivity by 15% compared to using just one bin. Both experiments do 
not see a significant excess (Fig. [TJ. The 90% confidence level (CL) limits calculated with a 
Bayesian method for a data sample of 2 fb~^ per experiment are B{B^ -^ fi^fi~) < 7.5 x 10~^ 
(DO'P and B{B^ -^ A*"^/^") < 4.7 x 10"^ (CDF^. Because of the good mass resolution of the 
tracking system CDF is able to separate B^ and B^ mesons and to quote a 90% CL limit on the 
5° decay of i3(S0 -^ /x+Zi") < 1.5 x 10"^ 
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Figure 1: Invariant mass spectrum of B^ -^ fJ-^ 1^ candidates measured by DO in two run ranges (left) and by 

CDF in three bins of neural network output (right). 
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The SM box and penguin processes of the D" — > fi fJ.~ decay are much more suppressed by 
the GIM mechanism than in the B^ — > //"'"/U" case. Therefore long distance processes like the 
decav via hadronic resonances and photons are dominant resulting in a predicted branching 
raticPof B{D^ -^ fi^fi^) > 4 x 10^^^. While no significant enhancement is expected in i?-parity 
conserving SUSY models, branching ratios up to 10~^ are possible if i2-parity is violated. 

CDF selects candidates of D^ -^ A*^/^" decays by a trigger on displaced tracks which allows 
to use the D^ — > 7r"'"7r~ mode for normalization. Muons are identified using muon chambers in the 
pseudorapidity ranges |r/| < 0.6 (CMU) and 0.6 < |7y| < 1 (CMX). Background events are reduced 
by requiring the D^ to come from a D* decay and by cutting on a lifetime information based 
probability ratio between signal events and B — > iJ,'^n~X events, the dominant background. In 
a data sample of 360 pb~^ the observed numbers of events of 3, and 1 in the acceptance regions 
CMU-CMU, CMU-CMX and CMX-CMX are consistent with the background expectations of 



4.9 ± 1.5, 2.7 ± 1.0 and 1.0 ± 0.5, respectively (Fig. [2]). The obtained 90% CL Bayesian limiP is 
i3(L>° -^ f^+fi-) < 4.3 X 10-^. 
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Figure 2: Invariant mass spectrum „. „, ■.+ + - . rr +- i-i.-.i 
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mass region (left) and outside the (j) mass region (right). 
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. x —f n^iJ.'^n^ decay is, like the D^ -^ ^^/^^ decay, dominated by long distance processes. 
But the size of this contribution depends on the dimuon mass. The selection of non-resonant 
dimuon masses therefore increases the sensitivity to new physics contributions, in particular 
from i?-parity violating SUSY. 

In the first part of the analysis DO selects events with m(/x^/i^) in the (p meson mass region 
to establish a resonant decay signal. In the m['K^ ^'^ ^i") spectrum a Df signal and a D~^ signal 
are observed (Fig. [3]) with a statistical significance of So" for both combined and 4.1o" for the 



D^ alone. The measured valu^of B{D^ 



67r+ -^ fi+fi-TT+) = (1.8 ± 0.5 ± 0.6) X 10-^ using 
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the resonant Df decay as normalization, is in good agreement with-- B{D^ 
11+ 11-) = (1.86 ± 0.26) X 10-6. 

For the search for the non-resonant decay events with m[^+ jjl^) in the (f) mass region are 
excluded. The observed number of 19 events in the m{'K+fi+fi^) search window is consistent 
with the background expectation of 25.8 it 4.6 events (Fig. [3]). By normalizing to the resonant 
D+ decay a 90% CL Bayesian limit of B{D+ — >■ tt+ii+ii~) < 3.9 x 10"^ is determined from a 
data sample of 1.3 fb^^. 
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The decays A^ -^ pn^ and Aj) -^ pK~ are allowed at tree level in the SM, but are suppressed by 
the small value of the involved CKM matrix element Vub- Therefore loop diagram processes can 
contribute at a magnitude that is comparable to the tree diagram process. The interference of 
these amplitudes can lead to a sizeable direct CP violation. In the SM an Aqp value of 0(10%) 
is predicted. While -R-parity violating SUSY processes would enhance the branching ratio from 
0(10"^) up to 0(10"^) they would at the same time reduce Aqp by one order of magnitude. 

Both rare A^ decays were first observed in a CDF analysis of rare B meson decays to two 
light hadroniD . The analysis technique developed there is reused here. It involves an unbinned 
likelihood fit of invariant mass under dipion hypothesis (Fig. |3]left), relations between daughter 
particle momenta, and particle identification information provided by the specific ionization 
energy loss in the tracker. With the decay B^ -^ K+it~ as normalization the quantities B{A^ — > 
P7T~)/B{B^ -^ K+Tr~) ■ /ao//bo = 0.0415 ± 0.0074 ± 0.0058 and B{A^ -^ pK-)/B{B° -^ 



K+TT-) ■ fAo/fso = 0.0663 ± 0.0089 ± 0.0084 are measure(P in a data sample of 1 fb"^ where 

b 

/ao//b" is the A? to B^ production ratio. Taking the production ratio measured by CDF^ 

b , , 

and the known B^ — > K'^7r~ branching ratio one obtains absolute branching ratios of B{A^ -^ 
p-K~) = (1.4 ± 0.3t[!J) X 10-6 and -B(Ag -^ pR-) = (2.2 ± 0.3tJ:8) x 10"^ which are in good 
agreement with the SM predictions of 1 x 10"^ and 2 x lO^^, respectively^. 

To measure the CP asymmetry Acp{I^ —^ ph~ , h = tt or K) = [B{K^ — > ph^) — B{K^ — > 
ph~^)\/[B{h^ -^ ph^) + B{A^ — s- ph'^)] the relative efficiencies are determined from inclusive 
A° -^ p-K' and A° -^ p7r+ decays. While the result of ^cp(A° ^ p-^~) = 0.03 ± 0.17 ± 0.05 is 
well consistent with no CP asymmetry, the value of Acp{J^ — »■ pK~) = 0.37 it 0.17 it 0.03 is 
about 2a away from zero. Fig. [4] illustrates the asymmetry as well as the good description of 
the data by the fit and the powerful A{)/A|] separation. 
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Figure 4: Invariant mass spectrum for tt^-k mass assignment (left) and relative probability density function 

(pdf) oiAl^pK-: pdf(A0)/[pdf(A2)+pdf(A«)] (right). 



3 Conclusions 

New world's best limits on the branching ratio of the rare decays B'^ -^ /i^/i", B^ -^ ii^fi~, 
D^ — > jx^ iJi~ and D^ -^ 7r~^fi'^fi~ were presented. Furthermore the first branching ratio and CP 
asymmetry measurement of charmless hadronic A^ decays were shown. These Tevatron results 
can impose stringent constraints on physics beyond the SM. A further significant reduction of 
the new physics models parameter space can be expected as more data is taken and analyzed. 
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